Group items tagged

The Common Crawl Foundation’s repository of openly and freely accessible web crawl data is about to go live as a Public Data Set on Amazon Web Services.

Elbaz’ goal in developing the repository: “You can’t access, let alone download, the Google or the Bing crawl data. So certainly we’re differentiated in being very open and transparent about what we’re crawling and actually making it available to developers,” he says. “You might ask why is it going to be revolutionary to allow many more engineers and researchers and developers and students access to this data, whereas historically you have to work for one of the big search engines…. The question is, the world has the largest-ever corpus of knowledge out there on the web, and is there more that one can do with it than Google and Microsoft and a handful of other search engines are already doing? And the answer is unquestionably yes. ”

Common Crawl’s data already is stored on Amazon’s S3 service, but now Amazon will be providing the storage space for free through the Public Data Set program. Not only does that remove from Common Crawl the storage burden and costs for hosting its crawl of 5 billion web pages – some 50 or 60 terabytes large – but it should make it easier for users to access the data, and remove the bandwidth-related costs they might incur for downloads. Users won’t have to deal with setting up accounts, being responsible for bandwidth bills incurred, and more complex authentication processes.

BitTorrent today released folder syncing software that replicates files across multiple computers using the same peer-to-peer file sharing technology that powers BitTorrent clients. The free BitTorrent Sync application is labeled as being in the alpha stage, so it's not necessarily ready for prime-time, but it is publicly available for download and working as advertised on my home network. BitTorrent, Inc. (yes, there is a legitimate company behind BitTorrent) took to its blog to announce the move from a pre-alpha, private program to the publicly available alpha. Additions since the private alpha include one-way synchronization, one-time secrets for sharing files with a friend or colleague, and the ability to exclude specific files and directories.

BitTorrent Sync provides "unlimited, secure file-syncing," the company said. "You can use it for remote backup. Or, you can use it to transfer large folders of personal media between users and machines; editors and collaborators. It’s simple. It’s free. It’s the awesome power of P2P, applied to file-syncing." File transfers are encrypted, with private information never being stored on an external server or in the "cloud." "Since Sync is based on P2P and doesn’t require a pit-stop in the cloud, you can transfer files at the maximum speed supported by your network," BitTorrent said. "BitTorrent Sync is specifically designed to handle large files, so you can sync original, high quality, uncompressed files."

Europe’s highest court on Tuesday struck down an international agreement that allowed companies to move digital information like people’s web search histories and social media updates between the European Union and the United States. The decision left the international operations of companies like Google and Facebook in a sort of legal limbo even as their services continued working as usual.The ruling, by the European Court of Justice, said the so-called safe harbor agreement was flawed because it allowed American government authorities to gain routine access to Europeans’ online information. The court said leaks from Edward J. Snowden, the former contractor for the National Security Agency, made it clear that American intelligence agencies had almost unfettered access to the data, infringing on Europeans’ rights to privacy. The court said data protection regulators in each of the European Union’s 28 countries should have oversight over how companies collect and use online information of their countries’ citizens. European countries have widely varying stances towards privacy.

Data protection advocates hailed the ruling. Industry executives and trade groups, though, said the decision left a huge amount of uncertainty for big companies, many of which rely on the easy flow of data for lucrative businesses like online advertising. They called on the European Commission to complete a new safe harbor agreement with the United States, a deal that has been negotiated for more than two years and could limit the fallout from the court’s decision.

Some European officials and many of the big technology companies, including Facebook and Microsoft, tried to play down the impact of the ruling. The companies kept their services running, saying that other agreements with the European Union should provide an adequate legal foundation.But those other agreements are now expected to be examined and questioned by some of Europe’s national privacy watchdogs. The potential inquiries could make it hard for companies to transfer Europeans’ information overseas under the current data arrangements. And the ruling appeared to leave smaller companies with fewer legal resources vulnerable to potential privacy violations.

Internet censors have a new strategy in their bid to block applications and websites: pressuring the large cloud providers that host them. These providers have concerns that are much broader than the targets of censorship efforts, so they have the choice of either standing up to the censors or capitulating in order to maximize their business. Today’s internet largely reflects the dominance of a handful of companies behind the cloud services, search engines and mobile platforms that underpin the technology landscape. This new centralization radically tips the balance between those who want to censor parts of the internet and those trying to evade censorship. When the profitable answer is for a software giant to acquiesce to censors' demands, how long can internet freedom last? The recent battle between the Russian government and the Telegram messaging app illustrates one way this might play out. Russia has been trying to block Telegram since April, when a Moscow court banned it after the company refused to give Russian authorities access to user messages. Telegram, which is widely used in Russia, works on both iPhone and Android, and there are Windows and Mac desktop versions available. The app offers optional end-to-end encryption, meaning that all messages are encrypted on the sender's phone and decrypted on the receiver's phone; no part of the network can eavesdrop on the messages. Since then, Telegram has been playing cat-and-mouse with the Russian telecom regulator Roskomnadzor by varying the IP address the app uses to communicate. Because Telegram isn't a fixed website, it doesn't need a fixed IP address. Telegram bought tens of thousands of IP addresses and has been quickly rotating through them, staying a step ahead of censors. Cleverly, this tactic is invisible to users. The app never sees the change, or the entire list of IP addresses, and the censor has no clear way to block them all. A week after the court ban, Roskomnadzor countered with an unprecedented move of its own: blocking 19 million IP addresses, many on Amazon Web Services and Google Cloud. The collateral damage was widespread: The action inadvertently broke many other web services that use those platforms, and Roskomnadzor scaled back after it became clear that its action had affected services critical for Russian business. Even so, the censor is still blocking millions of IP addresses.

U.S. Department of Justice Civil Rights Division U.S. Department of Education Office for Civil Rights

The European Union and the U.K. opened formal antitrust investigations into Facebook Inc.’s FB -0.86% classified-ads service Marketplace, ramping up regulatory scrutiny for the company in Europe. Both the European Commission—the EU’s top antitrust enforcer—and the U.K.’s Competition and Markets Authority said Friday they are investigating whether Facebook repurposes data it gathers from advertisers who buy ads in order to give illegal advantages to its own services, including its Marketplace online flea market. The U.K. added that it is also investigating whether Facebook uses advertiser data to give similar advantages to its online-dating service. The two competition watchdogs said they would coordinate their investigations.

Separately on Friday, Germany’s competition regulator announced that it is opening an investigation into Google’s News Showcase, in which the tech company pays to license certain content from news publishers. That probe, which is based on new powers Germany had granted the regulator, will look among other things at whether Google is imposing unfair conditions on publishers and how it selects participants, the Federal Cartel Office said.

The three newly opened cases are part of a new wave of antitrust enforcement in Europe. The European Commission filed formal charges last month against Apple Inc. for allegedly abusing its control over the distribution of music-streaming apps, including Spotify Technology SA . In November, it filed formal charges against Amazon.com Inc. for allegedly using nonpublic data it gathers from third-party sellers to unfairly compete against them. Both companies denied wrongdoing. At the same time, the U.K.’s CMA has opened investigations into Google’s announcement that it will retire third-party cookies, a technology advertisers use to track web users, and whether Apple imposes anticompetitive conditions on some app developers, including the use of Apple’s in-app payment system, which is also the subject of a lawsuit in the U.S. In the EU, the European Commission has been investigating Facebook for more than a year on multiple fronts. Facebook and the Commission have squabbled over access to internal documents as part of those investigations.

Microsoft has joined forces with Taser to combine the Azure cloud platform with law enforcement management tools.

Taser’s Axon body camera data management software on Evidence.com will run on Azure and Windows 10 devices to integrate evidence collection, analysis, and archival features as set forth by the Federal Bureau of Investigation Criminal Justice Information Services (CJIS) Security Policy. As per the partnership, Taser will utilize Azure’s machine learning and computing technologies to store police data on Microsoft’s government cloud. In addition, redaction capabilities of Taser will be improved which will assist police departments that are subject to bulk data requests. Currently, Taser is operating on Amazon Web Services; however this deal may entice police departments to upgrade their technology, which in turn would drive up sales of Windows 10. This partnership comes after Taser was given a lucrative deal with the Los Angeles Police Department (LAPD) last year, who ordered 7,000 body cameras equipped with 800 Axom body cameras for their officers in response to the recent deaths of several African Americans at the hands of police.

In order to ensure Taser maintains a monopoly on police body cameras, the corporation acquired contracts with police departments all across the nation for the purchase of body cameras through dubious ties to certain chiefs of police. The corporation announced in 2014 that “orders for body cameras [has] soared to $24.6 million from October to December” which represents a 5-fold increase in profits from 2013. Currently, Taser is in 13 cities with negotiations for new contracts being discussed in 28 more. Taser, according to records and interviews, allegedly has “financial ties to police chiefs whose departments have bought the recording devices.” In fact, Taser has been shown to provide airfare and luxury hotels for chiefs of police when traveling for speaking engagements in Australia and the United Arab Emirates (UAE); and hired them as consultants – among other perks and deals. Since 2013, Taser has been contractually bound with “consulting agreements with two such chiefs’ weeks after they retired” as well as is allegedly “in talks with a third who also backed the purchase of its products.”